''FlyVIZ'': A Novel Display Device to Provide Humans with 360o Vision by Coupling Catadioptric Camera with HMD.

International audienceHave you ever dreamed of having eyes in the back of your head? In this paper we present a novel display device called FlyVIZ which enables humans to experience a real-time 360° vision of their surroundings for the first time. To do so, we combine a panoramic image acquisition system (positioned on top of the user's head) with a Head-Mounted Display (HMD). The omnidirectional images are transformed to fit the characteristics of HMD screens. As a result, the user can see his/her surroundings, in real-time, with 360° images mapped into the HMD field-of- view. We foresee potential applications in different fields where augmented human capacity (an extended field-of-view) could benefit, such as surveillance, security, or entertainment. FlyVIZ could also be used in novel perception and neuroscience studies

Design and Evaluation of Methods to Prevent Frame Cancellation in Real-Time Stereoscopic Rendering

International audienceFrame cancellation comes from the conflict between two depth cues: stereo disparity and occlusion with the screen border. When this conflict occurs, the user suffers from poor depth perception of the scene. It also leads to uncomfortable viewing and eyestrain due to problems in fusing left and right images. In this paper we propose a novel method to avoid frame cancellation in real-time stereoscopic rendering. To solve the disparity/frame occlusion conflict, we propose rendering only the part of the viewing volume that is free of conflict by using clipping methods available in standard real-time 3D APIs. This volume is called the "Stereo Compatible Volume" (SCV) and the method is named "Stereo Compatible Volume Clipping" (SCVC). Black Bands, a proven method initially designed for stereoscopic movies is also implemented to conduct an evaluation. Twenty two people were asked to answer open questions and to score criteria for SCVC, Black Bands and a Control method with no specific treatment. Results show that subjective preference and user's depth perception near screen edge seem improved by SCVC, and that Black Bands did not achieve the performance we expected. At a time when stereoscopic capable hardware is available from the mass consumer market, the disparity/frame occlusion conflict in stereoscopic rendering will become more noticeable. SCVC could be a solution to recommend. SCVC's simplicity of implementation makes the method able to target a wide range of rendering software from VR application to game engine

Contribution à l'étude de la visualisation d'environnements réels et virtuels avec un champ visuel étendu

Who have never wanted to have eyes in the back of his head? This doctoral thesis proposes to study the extension of the human field-of-view (FoV) in both real and virtual environments. First we have designed FlyVIZ, a new device to increase the human FoV. It is composed of a helmet, combining a catadioptric camera, a HMD and an image processing algorithm. Wearing this device allows a user to experience 360° vision of its surroundings. The prototype is demonstrated through scenarii such as grasping an object held out behind their back without turning their head or walking backward through doorways. Then we have proposed a novel method to render virtual environments with wide FoV in real-time. To solve the rendering issue induced by usage of non-planar projections, we introduce a special stage in real-time rendering pipeline. Our method was then adapted for real-time stereoscopic rendering with 360° FoV. We have conducted a preliminary evaluation of real-time wide FoV rendering for a navigation task in virtual reality. Our results confirm that using a wide FoV rendering method could lead to more efficient navigation in terms of average task completion time. Among the different tested non-planar projection methods, the subjective preference is given to equirectangular and Hammer projections. We also address the problem of frame cancellation, generated by the conflict between two depth cues: stereo disparity and occlusion with the screen border. We have proposed the Stereoscopy Compatible Volume Clipping (SCVC), solving the problem by rendering only the part of the viewing volume free of disparity - frame occlusion conflict. The method was evaluated and results have shown that SCVC notably improved users’ depth perception and that the users expressed preference for SCVC. Wide FoV opens novel perspectives for environments exploration or monitoring. Therefore, it could benefit to several applications, both in real world context or virtual environments. In safety and security applications, firemen, policemen or soldiers could take advantage of wide FoV. Performance of searching task and fast exploration in virtual environments could also be improved with wide FoV.Qui n’a jamais souhaité avoir des yeux derrière la tête ? Cette thèse propose d’étudier l’extension du champ visuel humain, que ce soit dans le monde réel ou dans un environnement virtuel. Nous avons d’abord conçu FlyVIZ, un dispositif qui permet d’augmenter le champ visuel. Il est composé d’une caméra catadioptrique, d’un visiocasque et d’un algorithme de traitement d’image. Lorsqu’un utilisateur porte ce dispositif, il dispose d’une vue à 360 degrés de son environnement. Le prototype a été testé avec succès dans différents scenarios, comme attraper un objet tendu dans le dos sans tourner la tête, ou passer des portes en marchant à reculons. Ensuite nous avons proposé une nouvelle méthode pour le rendu d’environnements virtuels, avec un champ visuel étendu, et en temps réel. Pour résoudre les problèmes dus à l’utilisation de projections non planes, nous avons ajouté une nouvelle étape dans le pipeline graphique. Notre méthode a ensuite été adaptée au rendu stéréoscopique avec un champ visuel de 360°. Nous avons mené une évaluation préliminaire sur l’utilisation d’un champ visuel étendu appliqué à une tache de navigation dans des environnements virtuels. Nos résultats semblent confirmer que l’utilisation d’un champ visuel étendu permet une navigation plus efficace, en diminuant le temps moyen pour effectuer une tache. Parmi les différentes projections non planes testées, une préférence pour les projections équirectangulaire et de Hammer a été exprimée. Nous avons également traité le problème de conflit d’indice de profondeurs rencontré dans les images stéréoscopiques lorsqu’un objet affiché en parallaxe négative est partiellement occulté par un bord du support d’affichage. Nous avons proposé SCVC (Stereoscopy Compatible Volume Clipping, le découpage de la scène selon le volume compatible avec la stéréoscopie) pour résoudre ce problème en n’affichant que la partie de l’espace qui n’est pas sujette au conflit d’indice de profondeur. La méthode a été évaluée et les résultats ont montrés que SCVC améliore significativement la perception de la profondeur et que les utilisateurs expriment une préférence pour cette méthode. L’extension du champ visuel humain ouvre de nouvelles perspectives pour l’exploration ou la surveillance de l’environnement d’un utilisateur. Cette extension pourrait bénéficier à diverses applications, que ce soit dans le contexte d’un environnement réel ou virtuel. Pour la sécurité des personnes ou la défense, des pompiers, des policiers ou des soldats pourraient tirer avantage d’un champ visuel étendu. Ce type de visualisation peut également profiter à l’exploration rapide d’environnements virtuels ou à la recherche d’objet dans ces derniers

Dispositif de visualisation adapté à fournir un champvisuel étendu

L'invention concerne un dispositif de visualisation (2) adapté à être porté par un utilisateur et à fournir à l'utilisateur un champ visuel étendu. Le dispositif selon l'invention comporte : - un périphérique d'affichage (4) comportant des moyens d'affichage (6) adaptés à être positionnés devant les yeux de l'utilisateur, - un dispositif d'acquisition d'images (10) positionné sur un support (8) qui est adapté à être porté par l'utilisateur, ledit dispositif d'acquisition d'images (10) étant apte à fournir simultanément une pluralité de vues de l'environnement de l'utilisateur, comportant au moins une vue avant correspondant au champ visuel de l'utilisateur et une autre vue, au moins partiellement hors du champ visuel de l'utilisateur, au même instant temporel, - des moyens de transmission (16) des images acquises à une unité de traitement (18) apte à fournir des images traitées, chaque image traitée correspondant à une projection desdites vues dans un plan d'affichage, et - des moyens de restitution des images traitées sur lesdits moyens d'affichage (6)

Increasing Optical Tracking Workspace of VR Applications using Controlled Cameras

International audienceWe propose an approach to greatly increase the tracking workspace of VR applications without adding new sensors. Our approach relies on controlled cameras able to follow the tracked markers all around the VR workspace providing 6DoF tracking data. We designed the proof-of-concept of such approach based on two consumer-grade cameras and a pan-tilt head. The resulting tracking workspace could be greatly increased depending on the actuators' range of motion. The accuracy error and jitter were found to be rather limited during camera motion (resp. 0.3cm and 0.02cm). Therefore, whenever the final VR application does not require a perfect tracking accuracy over the entire workspace, we recommend using our approach in order to enlarge the tracking workspace

Stereoscopic Rendering of Virtual Environments with Wide Field-of-Views up to 360{\textdegree}

International audienceIn this paper we introduce a novel approach for stereoscopic ren- dering of virtual environments with a wide Field-of-View (FoV) up to 360◦. Handling such a wide FoV implies the use of non- planar projections and generates specific problems such as for ras- terization and clipping of primitives. We propose a novel pre-clip stage specifically adapted to geometric approaches for which prob- lems occur with polygons spanning across the projection disconti- nuities. Our approach integrates seamlessly with immersive virtual reality systems as it is compatible with stereoscopy, head-tracking, and multi-surface projections. The benchmarking of our approach with different hardware setups could show that it is well compliant with real-time constraint, and capable of displaying a wide range of FoVs. Thus, our geometric approach could be used in various VR applications in which the user needs to extend the FoV and appre- hend more visual information

Navigating in Virtual Environments with 360o Omnidirectional Rendering

International audienceTypical field-of-view (fov) of visual feedback in virtual reality applications is generally limited. In some cases, e.g. in videogames, the provided fov can be artificially increased, using simple perspective projection methods. In this paper, we design and evaluate different visualization techniques, inspired by the cartography domain, for navigating in virtual environments (VE) with a 360° horizontal fov of the scene. We have conducted an evaluation of different methods compared to a rendering method of reference, i.e. a perspective projection, in a basic navigation task. Our results confirm that using any omnidirectional rendering method could lead to more efficient navigation in terms of average task completion time. Among the different 360° projection methods, the subjective preference was significantly given to a cylindrical projection method (equirectangular). Taken together, our results suggest that omnidirectional rendering could be used in virtual reality applications in which fast navigation or full and rapid visual exploration are important. They pave the way to novel kinds of visual cues and visual rendering methods in virtual reality

Design and Evaluation of Methods to Prevent Frame Cancellation in Real-Time Stereoscopic Rendering

International audienceFrame cancellation comes from the conflict between two depth cues: stereo disparity and occlusion with the screen border. When this conflict occurs, the user suffers from poor depth perception of the scene. It also leads to uncomfortable viewing and eyestrain due to problems in fusing left and right images. In this paper we propose a novel method to avoid frame cancellation in real-time stereoscopic rendering. To solve the disparity/frame occlusion conflict, we propose rendering only the part of the viewing volume that is free of conflict by using clipping methods available in standard real-time 3D APIs. This volume is called the "Stereo Compatible Volume" (SCV) and the method is named "Stereo Compatible Volume Clipping" (SCVC). Black Bands, a proven method initially designed for stereoscopic movies is also implemented to conduct an evaluation. Twenty two people were asked to answer open questions and to score criteria for SCVC, Black Bands and a Control method with no specific treatment. Results show that subjective preference and user's depth perception near screen edge seem improved by SCVC, and that Black Bands did not achieve the performance we expected. At a time when stereoscopic capable hardware is available from the mass consumer market, the disparity/frame occlusion conflict in stereoscopic rendering will become more noticeable. SCVC could be a solution to recommend. SCVC's simplicity of implementation makes the method able to target a wide range of rendering software from VR application to game engine

An Optical Tracking System based on Hybrid Stereo/Single-View Registration and Controlled Cameras

International audienceOptical tracking is widely used in robotics applications such as unmanned aerial vehicle (UAV) localization. Unfortunately, such systems require many cameras and are, consequently, expensive. In this paper, we propose an approach to considerably increase the optical tracking volume without adding cameras. First, when the target becomes no longer visible by at least two cameras we propose a single-view tracking mode which requires only one camera. Furthermore, we propose to rely on controlled cameras able to track the UAV all around the volume to provide 6DoF tracking data through multi-view registration. This is achieved by using a visual servoing scheme. The two methods can be combined in order to maximize the tracking volume. We propose a proof-of-concept of such an optical tracking system based on two consumer-grade cameras and a pan-tilt actuator and we used this approach on UAV localization